Missing data is poorly handled and reported in prediction model studies using machine learning: a literature review

OBJECTIVES: Missing data is a common problem during the development, evaluation, and implementation of prediction models. Although machine learning (ML) methods are often said to be capable of circumventing missing data, it is unclear how these methods are used in medical research. We aim to find out if and how well prediction model studies using machine learning report on their handling of missing data. STUDY DESIGN AND SETTING: We systematically searched the literature on published papers between 2018 and 2019 about primary studies developing and/or validating clinical prediction models using any supervised ML methodology across medical fields. From the retrieved studies information about the amount and nature (e.g. missing completely at random, potential reasons for missingness) of missing data and the way they were handled were extracted. RESULTS: We identified 152 machine learning-based clinical prediction model studies. A substantial amount of these 152 papers did not report anything on missing data (n = 56/152). A majority (n = 96/152) reported details on the handling of missing data (e.g., methods used), though many of these (n = 46/96) did not report the amount of the missingness in the data. In these 96 papers the authors only sometimes reported possible reasons for missingness (n = 7/96) and information about missing data mechanisms (n = 8/96). The most common approach for handling missing data was deletion (n = 65/96), mostly via complete-case analysis (CCA) (n = 43/96). Very few studies used multiple imputation (n = 8/96) or built-in mechanisms such as surrogate splits (n = 7/96) that directly address missing data during the development, validation, or implementation of the prediction model. CONCLUSION: Though missing values are highly common in any type of medical research and certainly in the research based on routine healthcare data, a majority of the prediction model studies using machine learning does not report sufficient information on the presence and handling of missing data. Strategies in which patient data are simply omitted are unfortunately the most often used methods, even though it is generally advised against and well known that it likely causes bias and loss of analytical power in prediction model development and in the predictive accuracy estimates. Prediction model researchers should be much more aware of alternative methodologies to address missing data

Acoustic Characterization of a Vessel-on-a-Chip Microfluidic System for Ultrasound-Mediated Drug Delivery

Ultrasound in the presence of gas-filled microbubbles can be used to enhance local uptake of drugs and genes. To study the drug delivery potential and its underlying physical and biological mechanisms, an in vitro vessel model should ideally include 3D cell culture, perfusion flow, and membranefree soft boundaries. Here, we propose an organ-on-a-chip microfluidic platform to study ultrasound-mediated drug delivery: the OrganoPlate. The acoustic propagation into the OrganoPlate was determined to assess the feasibility of controlled microbubble actuation, which is required to study the microbubble-cell interaction for drug delivery. The pressure field in the OrganoPlate was characterized non-invasively by studying experimentally the well-known response of microbubbles and by simulating the acoustic wave propagation in the system. Microbubble dynamics in the OrganoPlate were recorded with the Brandaris 128 ultrahigh speed camera (17 Mfps) and a control experiment was performed in an OptiCell, an in vitro monolayer cell culture chamber that is conventionally used to study ultrasound-mediated d

Opening of endothelial cell–cell contacts due to sonoporation

Ultrasound insonification of microbubbles can locally increase vascular permeability to enhance drug delivery. To control and optimize the therapeutic potential, we need to better understand the underlying biological mechanisms of the drug delivery pathways. The aim of this in vitro study was to elucidate the microbubble-endothelial cell interaction using the Brandaris 128 ultra-high-speed camera (up to 25 Mfps) coupled to a custom-built Nikon confocal microscope, to visualize both microbubble oscillation and the cellular response. Sonoporation and opening of cell-cell contacts by single αVβ3-targeted microbubbles (n = 152) was monitored up to 4 min after ultrasound insonification (2 MHz, 100–400 kPa, 10 cycles). Sonoporation occurred when microbubble excursion amplitudes exceeded 0.7 μm. Quantification of the influx of the fluorescent model drug propidium iodide upon sonoporation showed that the size of the created pore increased for larger microbubble excursion amplitudes. Microbubble-mediated opening of cell-cell contacts occurred as a cellular response upon sonoporation and did not correlate with the microbubble excursion amplitude itself. The initial integrity of the cell-cell contacts affected the susceptibly to drug delivery, since cell-cell contacts opened more often when cells were only partially attached to their neighbors (48%) than when fully attached (14%). The drug delivery outcomes were independent of nonlinear microbubble behavior, microbubble location, and cell size. In conclusion, by studying the microbubble–cell interaction at nanosecond and nanometer resolution the relationship between drug delivery pathways and their underlying mechanisms was further unraveled. These novel insights will aid the development of safe and efficient microbubble-mediated drug delivery

High-Resolution Imaging of Intracellular Calcium Fluctuations Caused by Oscillating Microbubbles

Ultrasound insonification of microbubbles can locally enhance drug delivery, but the microbubble–cell interaction remains poorly understood. Because intracellular calcium (Cai 2+) is a key cellular regulator, unraveling the Cai 2+ fluctuations caused by an oscillating microbubble provides crucial insight into the underlying bio-effects. Therefore, we developed an optical imaging system at nanometer and nanosecond resolution that can resolve Cai 2+ fluctuations and microbubble oscillations. Using this system, we clearly distinguished three Cai 2+ uptake profiles upon sonoporation of endothelial cells, which strongly correlated with the microbubble oscillation amplitude, severity of sonoporation and opening of cell–cell contacts. We found a narrow operating range for viable drug delivery without lethal cell damage. Moreover, adjacent cells were affected by a calcium wave propagating at 15 μm/s. With the unique optical system, we unraveled the microbubble oscillation behavior required for drug delivery and Cai 2+ fluctuations, providing new insight into the microbubble–cell interaction to aid clinical translation

Combined Confocal Microscope and Brandaris 128 Ultra-High-Speed Camera

Controlling microbubble-mediated drug delivery requires the underlying biological and physical mechanisms to be unraveled. To image both microbubble oscillation upon ultrasound insonification and the resulting cellular response, we developed an optical imaging system that can achieve the necessary nanosecond temporal and nanometer spatial resolutions. We coupled the Brandaris 128 ultra-high-speed camera (up to 25 million frames per second) to a custom-built Nikon A1R+ confocal microscope. The unique capabilities of this combined system are demonstrated with three experiments showing microbubble oscillation leading to either endothelial drug delivery, bacterial biofilm disruption, or structural changes in the microbubble coating. In conclusion, using this state-of-the-art optical imaging system, microbubble-mediated drug delivery can be studied with high temporal resolution to resolve microbubble oscillation and high spatial resolution and detector sensitivity to discern cellular response. Combining these two imaging technologies will substantially advance our knowledge on microbubble behavior and its role in drug delivery

Local Drug Delivery by Microbubbles

The aim of this thesis was to unravel the microbubble-cell interaction required for local drug delivery. By developing methodological innovations for novel in vitro studies, the underlying biophysical mechanisms of ultrasound-mediated drug delivery were further elucidated

Ligand Distribution and Lipid Phase Behavior in Phospholipid-Coated Microbubbles and Monolayers

Phospholipid-coated targeted microbubbles are ultrasound contrast agents that can be used for molecular imaging and enhanced drug delivery. However, a better understanding is needed of their targeting capabilities and how they relate to microstructures in the microbubble coating. Here, we investigated the ligand distribution, lipid phase behavior, and their correlation in targeted microbubbles of clinically relevant sizes, coated with a ternary mixture of 1,2- dipalmitoyl-sn-glycero-3-phosphocholine (DPPC) or 1,2-distearoyl-sn-glycero-3- phosphocholine (DSPC), with PEG40-stearate and DSPE-PEG2000. To investigate the effect of lipid handling prior to microbubble production in DSPC-based microbubbles, the components were either dispersed in aqueous medium (direct method) or first dissolved and mixed in an organic solvent (indirect method). To determine the lipid-phase behavior of all components, experiments were conducted on monolayers at the air/water interface. In comparison to pure DSPC and DPPC, the ternary mixtures had an additional transition plateau around 10−12 mN/m. As confirmed by infrared reflection absorption spectroscopy (IRRAS), this plateau was due to a transition in the conformation of the PEGylated components (mushroom to brush). While the condensed phase domains had a different morphology in the ternary DPPC and DSPC monolayers on the Langmuir trough, the domain morphology was similar in the coating of both ternary DPPC and DSPC microbubbles (1.5−8 μm diameter). The ternary DPPC microbubbles had a homogenous ligand distribution and significantly less liquid condensed (LC) phase area in their coating than the DSPC-based microbubbles. For ternary DSPC microbubbles, the ligand distribution and LC phase area in the coating depended on the lipid handling. The direct method resulted in a heterogeneous ligand distribution, less LC phase area than the indirect method, and the ligand colocalizing with the liquid expanded (LE) phase area. The indirect method resulted in a homogenous ligand distribution with the largest LC phase area. In conclusion, lipid handling prior to microbubble production is of importance for a ternary mixture of DSPC, PEG40-stearate, and DSPE-PEG2000